Simulating bout-and-pause patterns with reinforcement learning.

Animal responses occur according to a specific temporal structure composed of two states, where a bout is followed by a long pause until the next bout. Such a bout-and-pause pattern has three components: the bout length, the within-bout response rate, and the bout initiation rate. Previous studies have investigated how these three components are affected by experimental manipulations. However, it remains unknown what underlying mechanisms cause bout-and-pause patterns. In this article, we propose two mechanisms and examine computational models developed based on reinforcement learning. The model is characterized by two mechanisms. The first mechanism is choice-an agent makes a choice between operant and other behaviors. The second mechanism is cost-a cost is associated with the changeover of behaviors. These two mechanisms are extracted from past experimental findings. Simulation results suggested that both the choice and cost mechanisms are required to generate bout-and-pause patterns and if either of them is knocked out, the model does not generate bout-and-pause patterns. We further analyzed the proposed model and found that it reproduced the relationships between experimental manipulations and the three components that have been reported by previous studies. In addition, we showed alternative models can generate bout-and-pause patterns as long as they implement the two mechanisms.

Robustness analysis of decoding SSVEPs in humans with head movements using a moving visual flicker.

OBJECTIVE: The emergence of mobile electroencephalogram (EEG) platforms have expanded the use cases of brain-computer interfaces (BCIs) from laboratory-oriented experiments to our daily life. In challenging situations where humans' natural behaviors such as head movements are unrestrained, various artifacts could deteriorate the performance of BCI applications. This paper explored the effect of muscular artifacts generated by participants' head movements on the signal characteristics and classification performance of steady-state visual evoked potentials (SSVEPs). APPROACH: A moving visual flicker was employed to induce not only SSVEPs but also horizontal and vertical head movements at controlled speeds, leading to acquiring EEG signals with intensity-manipulated muscular artifacts. To properly induce neck muscular activities, a laser light was attached to participants' heads to give visual feedback; the laser light indicates the direction of the head independently from eye movements. The visual stimulus was also modulated by four distinct frequencies (10, 11, 12, and 13 Hz). The amplitude and signal-to-noise ratio (SNR) were estimated to quantify the effects of head movements on the signal characteristics of the elicited SSVEPs. The frequency identification accuracy was also estimated by using well-established decoding algorithms including calibration-free and fully-calibrated approaches. MAIN RESULTS: The amplitude and SNR of SSVEPs tended to deteriorate when the participants moved their heads, and this tendency was significantly stronger in the vertical head movements than in the horizontal movements. The frequency identification accuracy also deteriorated in proportion to the speed of head movements. Importantly, the accuracy was significantly higher than its chance-level regardless of the level of artifact contamination and algorithms. SIGNIFICANCE: The results suggested the feasibility of decoding SSVEPs in humans freely moving their head directions, facilitating the real-world applications of mobile BCIs.

Non-standard trajectories found by machine learning for evaporative cooling of 87Rb atoms.

We present a machine-learning experiment involving evaporative cooling of gaseous 87Rb atoms. The evaporation trajectory was optimized to maximize the number of atoms cooled down to a Bose-Einstein condensate using Bayesian optimization. After 300 trials within 3 hours, Bayesian optimization discovered trajectories that achieved atom numbers comparable with those of manual tuning by a human expert. Analysis of the machine-learned trajectories revealed minimum requirements for successful evaporative cooling. We found that the manually obtained curve and the machine-learned trajectories were quite similar in terms of evaporation efficiency, although the manual and machine-learned evaporation ramps were significantly different.

Transfer Learning Over Time and Position in Wearable Myoelectric Control Systems.

Wearable sensors for upper limbs enable the use of myoelectric control systems in real environments. An important issue in the practical use of myoelectric control is how to deal with the variations of electromyograms (EMGs); the distribution of EMGs changes over days and device (electrode) positions. The amount of training data is usually limited, as the data are collected at the beginning of the system use. To compensate for the difference of EMGs over time and device placement with limited-amount training data, transfer learning can be employed. However, it was unclear how transfer learning improve the motion recognition accuracy over long-term use with varying device positions. In this paper, we evaluated transfer learning algorithms on one-month long data with three different device positions. We found that transfer learning was able to compensate for the variations over long period and also over different electrode placements, suggesting the practical efficacy of transfer learning. But there were some cases when transfer learning did not recover the original accuracy, in particular when electrodes were placed at "out-of-muscle" positions. These findings would motivate further investigations into the design of myoelectric control systems, e.g., denser electrode configurations or lifetime-long recordings.

Imputing Missing Values in EEG with Multivariate Autoregressive Models.

Wearable measurement for electroencephalogram (EEG) is expected to enable brain-computer interfaces, biomedical engineering, and neuroscience studies in real environments. When wearable devices are in practical use, only the user (subject) can take care of measurement, unlike laboratory- oriented experiments, where experimenters are always with the subject. As a result, measurement troubles such as artifact contamination or electrode impairment cannot be easily corrected, and EEG recordings will become incomplete, including many missing values. If the missing values are imputed (interpolated) and complete data without missing entries are available, we can employ existing signal analysis techniques that assume compete data. In this paper, we propose an EEG signal imputation method based on multivariate autoregressive (MAR) modeling and its iterative estimation and simulation, inspired by the multiple imputation procedure. We evaluated the proposed method with real data with artificial missing entries. Experimental results show that the proposed method outperforms popular baseline interpolation methods. Our iterative scheme is simple yet effective, and can be the foundation for many extensions.

Semi-simulation Experiments for Quantifying the Performance of SSVEP-based BCI after Reducing Artifacts from Trapezius Muscles.

Muscular artifacts often contaminate electroencephalograms (EEGs) and deteriorate the performance of brain-computer interfaces (BCIs). Although many artifact reduction techniques are available, most of the studies have focused on their reduction ability (i.e. reconstruction errors), and it has been missing to evaluate their effect on the performance of BCIs. This study aims at evaluating the performance of a state-of-the-art muscular artifact reduction technique on a scenario of a steady-state visual evoked potentials (SSVEPs)based BCI. The performance was evaluated based on a semisimulation setting using a benchmark dataset of SSVEPs artificially contaminated by muscular artifacts acquired from the trapezius. Our results showed that combining the artifact reduction method and the classification algorithm based on the task-related component analysis gained improved classification accuracy. Interestingly, the artifact reduction setting minimizing the reconstruction errors, i.e. elaborately recovering the true EEG waveforms, was inconsistent to the one maximizing the classification performance. The results suggest that artifact reduction methods should be tuned so as to tomaximize performance of BCIs.

Learning a common dictionary for subject-transfer decoding with resting calibration.

Brain signals measured over a series of experiments have inherent variability because of different physical and mental conditions among multiple subjects and sessions. Such variability complicates the analysis of data from multiple subjects and sessions in a consistent way, and degrades the performance of subject-transfer decoding in a brain-machine interface (BMI). To accommodate the variability in brain signals, we propose 1) a method for extracting spatial bases (or a dictionary) shared by multiple subjects, by employing a signal-processing technique of dictionary learning modified to compensate for variations between subjects and sessions, and 2) an approach to subject-transfer decoding that uses the resting-state activity of a previously unseen target subject as calibration data for compensating for variations, eliminating the need for a standard calibration based on task sessions. Applying our methodology to a dataset of electroencephalography (EEG) recordings during a selective visual-spatial attention task from multiple subjects and sessions, where the variability compensation was essential for reducing the redundancy of the dictionary, we found that the extracted common brain activities were reasonable in the light of neuroscience knowledge. The applicability to subject-transfer decoding was confirmed by improved performance over existing decoding methods. These results suggest that analyzing multisubject brain activities on common bases by the proposed method enables information sharing across subjects with low-burden resting calibration, and is effective for practical use of BMI in variable environments.

Decoding spatial attention by using cortical currents estimated from electroencephalography with near-infrared spectroscopy prior information.

For practical brain-machine interfaces (BMIs), electroencephalography (EEG) and near-infrared spectroscopy (NIRS) are the only current methods that are non-invasive and available in non-laboratory environments. However, the use of EEG and NIRS involves certain inherent problems. EEG signals are generally a mixture of neural activity from broad areas, some of which may not be related to the task targeted by BMI, hence impairing BMI performance. NIRS has an inherent time delay as it measures blood flow, which therefore detracts from practical real-time BMI utility. To try to improve real environment EEG-NIRS-based BMIs, we propose here a novel methodology in which the subjects' mental states are decoded from cortical currents estimated from EEG, with the help of information from NIRS. Using a Variational Bayesian Multimodal EncephaloGraphy (VBMEG) methodology, we incorporated a novel form of NIRS-based prior to capture event related desynchronization from isolated current sources on the cortical surface. Then, we applied a Bayesian logistic regression technique to decode subjects' mental states from further sparsified current sources. Applying our methodology to a spatial attention task, we found our EEG-NIRS-based decoder exhibited significant performance improvement over decoding methods based on EEG sensor signals alone. The advancement of our methodology, decoding from current sources sparsely isolated on the cortex, was also supported by neuroscientific considerations; intraparietal sulcus, a region known to be involved in spatial attention, was a key responsible region in our task. These results suggest that our methodology is not only a practical option for EEG-NIRS-based BMI applications, but also a potential tool to investigate brain activity in non-laboratory and naturalistic environments.

A state-space modeling approach for localization of focal current sources from MEG.

State-space modeling is a promising approach for current source reconstruction from magnetoencephalography (MEG) because it constrains the spatiotemporal behavior of inverse solutions in a flexible manner. However, state-space model-based source localization research remains underdeveloped; extraction of spatially focal current sources and handling of the high dimensionality of the distributed source model remain problematic. In this study, we propose a novel state-space model-based method that resolves these problems, extending our previous source localization method to include a temporal constraint by state-space modeling. To enable focal current reconstruction, we account for spatially inhomogeneous temporal dynamics by introducing dynamics model parameters that differ for each cortical position. The model parameters and the intensity of the current sources are jointly estimated according to a bayesian framework. We circumvent the high dimensionality of the problem by assuming prior distributions of the model parameters to reduce the sensitivity to unmodeled components, and by adopting variational bayesian inference to reduce the computational cost. Through simulation experiments and application to real MEG data, we have confirmed that our proposed method successfully reconstructs focal current activities, which evolve with their temporal dynamics.

Sparse bayesian learning of filters for efficient image expansion.

We propose a framework for expanding a given image using an interpolator that is trained in advance with training data, based on sparse bayesian estimation for determining the optimal and compact support for efficient image expansion. Experiments on test data show that learned interpolators are compact yet superior to classical ones.

Superresolution with compound Markov random fields via the variational EM algorithm.

This study deals with a reconstruction-type superresolution problem and the accompanying image registration problem simultaneously. We propose a Bayesian approach in which the prior is modeled as a compound Gaussian Markov random field (MRF) and marginalization is performed over unknown variables to avoid overfitting. Our algorithm not only avoids overfitting, but also preserves discontinuity in the estimated image, unlike existing single-layer Gaussian MRF models for Bayesian superresolution. Maximum-marginal-likelihood estimation of the registration parameters is carried out using a variational EM algorithm where hidden variables are marginalized out, and the posterior distribution is variationally approximated by a factorized trial distribution. High-resolution image estimates are obtained through the process of posterior computation in the EM algorithm. Experiments show that our Bayesian approach with the two-layer compound model exhibits better performance both in quantitative measures and visual quality than the single-layer model.